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Abstract 

This  paper  describes  an  application 
of  state-of-the-art  spoken  language 
technology  (OAA/Gemini/Nuance) 
to  a  new  problem  domain:  engaging 
students  in  automated  tutorial  dia¬ 
logues  in  order  to  evaluate  and  im¬ 
prove  their  performance  in  a  train¬ 
ing  simulator. 

1  Introdnction 

Shipboard  damage  control  refers  to  the  task  of 
containing  the  effects  of  fire,  explosions,  hull 
breaches,  flooding,  and  other  critical  events 
that  can  occur  aboard  Naval  vessels.  The 
high-stakes,  high-stress  nature  of  this  task,  to¬ 
gether  with  limited  opportunities  for  real-life 
training,  make  damage  control  an  ideal  target 
for  Al-enabled  educational  technologies  like 
training  simulators  and  tutoring  systems. 

This  paper  describes  the  spoken  dialogue 
system  we  developed  for  automated  critiquing 
of  student  performance  on  a  damage  control 
training  simulator.  The  simulator  is  DC- 
Train  (Bulitko  and  Wilkins,  1999),  an  im¬ 
mersive,  multimedia  training  environment  for 
damage  control.  DC-Train’s  training  sce¬ 
narios  simulate  a  mixture  of  physical  phenom¬ 
ena  (e.g.,  fire,  flooding)  and  personnel  issues 
(e.g.,  casualties,  communications,  standard¬ 
ized  procedures).  Our  current  tutoring  sys¬ 
tem  is  restricted  fire  damage  scenarios  only, 
and  in  particular  to  the  twelve  fire  scenar¬ 
ios  available  in  DC-Train  version  2.5,  but 
in  future  versions  we  plan  to  support  post¬ 
session  critiques  for  all  of  the  damage  phe¬ 
nomena  that  will  be  modeled  by  DC-Train 


4.0:  fire,  flooding,  missile  damage,  and  wall 
or  firemain  ruptures. 

2  Previous  Work 

Eliciting  self-explanation  from  a  student  has 
been  shown  to  be  a  highly  effective  tutoring 
method  (Chi  et  al.,  1994).  For  this  reason, 
a  number  of  automated  tutoring  systems  cur¬ 
rently  use  NLP  techniques  to  engage  students 
in  reflective  dialogues.  Three  notable  exam¬ 
ples  are  the  medical  CiRCSiM  tutor  (Zhou  et 
al.,  1999);  the  Basic  Electricity  and  Electron¬ 
ics  (BE&E)  tutor  (Rose  et  al.,  1999);  and 
the  computer  literacy  AutoTutor  (Wiemer- 
Hastings  et  al.,  1999). 

Our  system  shares  several  features  with 
these  three  tutoring  systems: 

A  knowledge  base  Our  system  encodes 
all  domain  knowledge  relevant  to  supporting 
intelligent  tutoring  feedback  into  a  structure 
called  an  Expert  Session  Summary  (Section 
4).  These  expert  summaries  encode  causal 
relationships  between  events  on  the  ship  as 
well  as  the  proper  and  improper  responses  to 
shipboard  crises. 

Tutoring  strategies  In  our  system,  as  in 
those  above,  the  flow  of  dialogue  is  controlled 
by  (essentially)  a  finite-state  transition  net¬ 
work  (Fig.  1). 

An  interpretation  component  In  our 

system,  the  student’s  speech  is  recognized  and 
parsed  into  logical  forms  (Section  3).  A  dia¬ 
logue  manager  inspects  the  current  dialogue 
information  state  to  determine  how  best  to 
incorporate  each  new  utterance  into  the  dia¬ 
logue  (Lemon  et  al.,  2001). 
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Figure  1:  Post-session  dialogue  move  graph  (simplified) 


However,  an  important  difference  is  that 
the  three  systems  above  are  entirely  text- 
based,  whereas  ours  is  a  spoken  dialogue  sys¬ 
tem.  Our  speech  interface  offers  greater  natu¬ 
ralness  than  keyboard-based  input.  In  this  re¬ 
spect,  our  system  is  similar  to  cove  (Roberts, 
2000),  a  training  simulator  for  conning  Navy 
ships  that  uses  speech  to  interact  with  the 
student.  But  whereas  COVE  uses  short  conver¬ 
sational  exchanges  to  coach  the  student  dur¬ 
ing  the  simulation,  our  system  engages  in  ex¬ 
tended  tutorial  dialogues  after  the  simulation 
has  ended.  Besides  being  more  natural,  spo¬ 
ken  language  systems  are  also  better  suited  to 
multimodal  interactions  (viz.,  one  can  point 
and  click  while  talking  but  not  while  typing). 
An  additional  significant  difference  between 
our  system  and  a  number  of  other  automated 
tutoring  systems  is  our  use  of  ‘deep’  process¬ 
ing  techniques.  While  other  systems  utilize 
‘shallow’  statistical  approaches  like  Latent  Se¬ 
mantic  Analysis  (e.g.  AutoTutor),  our  system 
utilizes  Gemini,  a  symbolic  grammar.  This 
approach  enables  us  to  provide  precise  and 
reliable  meaning  representations. 

3  Implementation 

To  facilitate  the  implementation  of  multi¬ 
modal,  mixed-initiative  tutoring  interactions, 
we  decided  to  implement  our  system  within 
the  Open  Agent  Architecture  (OAA)  (Martin 
et  ah,  1999).  OAA  is  a  framework  for  coor¬ 
dinating  multiple  asynchronous  communicat¬ 
ing  processes.  The  core  of  OAA  is  a  ‘facilita¬ 
tor’  which  manages  message  passing  between 
a  number  of  software  agents  that  specialize 
in  certain  tasks  (e.g.,  speech  recognition  or 


database  queries).  Our  system  uses  OAA  to 
coordinate  the  following  five  agents: 

1.  The  Gemini  NLP  system  (Dowding  et 
ah,  1993).  Gemini  uses  a  single  unifi¬ 
cation  grammar  both  for  parsing  strings 
of  words  into  logical  forms  (LFs)  and  for 
generating  sentences  from  LF  inputs. 

2.  A  Nuance  speech  recognition  server, 
which  converts  spoken  utterances  to 
strings  of  words.  The  Nnance  server  re¬ 
lies  on  a  language  model,  which  is  com¬ 
piled  directly  from  the  Gemini  grammar, 
ensuring  that  every  recognized  utterance 
is  assigned  an  LF. 

3.  The  Festival  text-to-speech  system, 
which  ‘speaks’  word  strings  generated  by 
Gemini. 

4.  A  Dialogue  Manager  which  coordi¬ 
nates  inputs  from  the  user,  interprets  the 
user’s  dialogue  moves,  updates  the  dia¬ 
logue  context,  and  delivers  speech  and 
graphical  outputs  to  the  user. 

5.  A  Critique  Planner,  described  below 
in  Section  4. 

Agents  1-3  are  reusable,  ‘off-the-shelf’  dia¬ 
logue  system  components  (apart  from  the 
Gemini  grammar,  which  must  be  modified  for 
each  application).  We  implemented  agents  4 
and  5  in  Java  specifically  for  this  application. 

Variants  of  this  OAA/Gemini/Nuance  ar- 
chitectnre  have  been  deployed  successfully  in 
other  dialogue  systems,  notably  SRI’s  Gom- 
mandTalk  (Stent  et  ah,  1999)  and  an  un- 


|AbQirt|  I  Exit  I 


iCSLI/MURI  DC-Train  tutor  dialog 


S 

File  Options  Help 

Load  summary:  [From  file]  |From  clipboarr^  ,  Critique: 


Session  summary: 


Restart  this  session 


Session:  4  fires 

9  Fire  in  2-174-4:  extinguished 

repair_five:  Set  fire  boundaries 
repair_five:  Fight  fire 
9  iii/  Fire  in  4-72-0:  extinguished 

(faii)(err,  comm)  repair_five:  Fight  fire 
No  Personnei 
^  repairjhree;  Fightfire 

repair_two:  Set  fire  boundaries 
®*  ^  Fire  in  -3-1 58-3:  not  extinguished 
Fire  in  2-41 4-0:  extinguished 


Speak 

i  should  have  sent  repair  two  to  set  fire  boundaries 

^sponse: 

what  actions  did  you  take? 

Repeat  ' 

— - - 

1  Dialogue  history 

luuui..  UJ.U  yuu  uu  uiLUiiij/ 

Student:  i  dont  know 

'  Tutor:  repair  five  was  still  fighting  the  earlier  fire 
Tutor:  a  repair  team  can  only  be  sent  to  one  compartment  at  a  time 
Tutor:  what  should  you  have  done  instead? 

Student:  i  should  have  sent  repair  two  to  set  fire  boundaries 
Tutor:  yes  thats  correct 
'Tutor:  okay  lets  continue 

Tutor:  the  third  fire  was  in  the  torpedo  magazine 
I  Tutor:  this  fire  was  never  extinguished 
I  Tutor:  what  actions  did  you  take? 


Figure  2:  Screen  shot  of  post-session  tutorial  dialogue  system 


manned  helicopter  interface  developed  in  our 
laboratory  (Lemon  et  ah,  2001). 

4  Planning  the  dialogne 

Each  stndent  session  with  DC-Train  pro¬ 
duces  a  session  transcript,  i.e.  a  time-stamped 
record  of  every  event  (both  compnter-  and 
student-initiated)  that  occnrred  during  the 
simulation.  These  transcripts  serve  as  the 
input  to  our  post-session  Critique  Planner 
(CP). 

The  CP  plans  a  post-session  tntorial  di¬ 
alogne  in  two  steps.  In  the  first  step,  an 
Expert  Session  Summary  (ESS)  is  cre¬ 
ated  from  the  session  transcript.  The  ESS 
is  a  tree  whose  parent  nodes  represent  dam¬ 
age  events  and  whose  leaves  represent  actions 
taken  in  response  to  those  damage  events. 
Each  student-initiated  action  in  the  ESS  is 
evaluated  as  to  its  timeliness  and  conformance 
to  damage  control  doctrine.  Actions  that  the 
student  shonld  have  taken  but  did  not  are  also 
inserted  into  the  ESS  and  flagged  as  such. 


Each  action  node  in  the  ESS  therefore  falls 
into  one  of  three  classes:  (i)  correct  actions; 
(ii)  errors  of  commission  (e.g.,  the  student 
sets  fire  containment  bonndaries  incorrectly); 
and  (hi)  errors  of  omission  (e.g.,  the  student 
fails  to  secnre  permission  from  the  captain  be¬ 
fore  flooding  certain  compartments). 

Our  current  tutoring  system  covers  scenar¬ 
ios  generated  by  DC-Train  2.5,  which  covers 
fire  scenarios  only.  Fntnre  versions  will  nse 
scenarios  generated  by  DC-Train  4.0,  which 
covers  damage  control  scenarios  involving  fire, 
smoke,  flooding,  pipe  and  hull  ruptures,  and 
equipment  deactivation.  Onr  current  tutor¬ 
ing  system  is  based  on  an  ESS  graph  that  is 
generated  by  an  expert  model  that  consists 
of  an  ad-hoc  set  of  firefighting  rules.  Future 
versions  will  be  based  on  an  ESS  graph  that 
is  generated  by  an  successor  to  the  Minerva- 
DCA  expert  model  (Bulitko  and  Wilkins, 
1999),  an  extended  Petri  Net  envisionment- 
based  reasoning  system.  The  new  expert 
model  is  designed  to  produce  an  ESS  graph 
dnring  the  conrse  of  problem  solving  that  con- 


tains  nodes  for  all  successful  and  unsuccessful 
plan  and  goal  achievement  events,  along  with 
an  explanation  structure  for  each  graph  node. 

The  second  step  in  planning  the  post¬ 
session  tutorial  dialogue  is  to  produce  a  di¬ 
alogue  move  graph  (Fig.  1).  This  is  a  di¬ 
rected  graph  that  encodes  all  possible  configu¬ 
rations  of  dialogue  structure  and  content  that 
can  be  handled  by  the  system. 

Generating  an  appropriate  dialogue  move 
graph  from  an  ESS  requires  pedagogical 
knowledge,  and  in  particular  a  tutoring  strat¬ 
egy.  The  tutoring  strategy  we  adopted  is 
based  on  our  analysis  of  videotapes  of  fifteen 
actual  DC- Train  post-session  critiques  con¬ 
ducted  by  instructors  at  the  Navy’s  Surface 
Warfare  Officer’s  School  in  Newport,  RI.  The 
strategy  we  observed  in  these  critiques,  and 
implemented  in  our  system,  can  be  outlined 
as  follows: 

1.  Summarize  the  results  of  the  simulation 
(e.g.,  the  final  condition  of  the  ship). 

2.  For  each  major  damage  event  in  the  ESS: 

(a)  Ask  the  student  to  review  his  ac¬ 
tions,  correcting  his  recollections  as 
necessary. 

(b)  Evaluate  the  correctness  of  each  stu¬ 
dent  action. 

(c)  If  the  student  committed  errors, 
ask  him  how  these  could  have  been 
avoided,  and  evaluate  the  correct¬ 
ness  of  his  responses. 

3.  Finally,  review  each  type  of  error  that 
arose  in  step  (2c). 

A  screen  shot  of  the  tutoring  system  in 
action  is  shown  in  Fig.  2.  As  soon  as  a 
DC-Train  simulation  ends,  the  dialogue  sys¬ 
tem  starts  up  and  the  dialogue  manager  be¬ 
gins  traversing  the  dialogue  move  graph.  As 
the  dialogue  unfolds,  a  graphical  representa¬ 
tion  of  the  ESS  is  revealed  to  the  student  in 
piecemeal  fashion  as  depicted  in  the  top  right 
frame  of  Fig.  2. 
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